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High-Speed Internet Processing 











TCP SYN 



GET/ 

User-Agent: Mozilla 4.1 , IE5 
Host:www.google.com 
Cookie :ik=xzxsrzczccz 



TCP FIN 



IGCH< 



09:28:01 2008-10-13 1 

09:28:13 2008-10-131 



1 7776 80 GET / Cookie: ik= qyzwww 

1 3456 80 GET / Cookie: ik= xzxsrzczccz 




Event data sent to bulk store 
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High-Speed Internet Processing 



• Bulk events key to SIGINT success on Internet 

• Event types that are valuable for Intelligence change (quickly) 

- 2000 SMTP/POP3 

- 2001 Webmail 

- 2007 vBulletin 

- 2008 Social Networks,...,? 

• GCHQ’s Applied Research are pioneering ways of dealing with this: 

- Presence Events (TDI) 

- Very large scale high speed flat file storage to bulk store TDIs 

- Just enough data marts 
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IP Packet Information 



• Many possible types of information 

• Many techniques available 

• HTTP Get requests dominate cutting edge 
techniques 

• To get Intelligence value Information must 



relate to a person or device... a TDI 
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Target 

Detection 

Identifier 



Who 

When 

Where 

(doing) What 
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Target 
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Fundamental atom of the Internet age. 
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Target Detection Identifiers 



• DEFINITION 

- TDIs are definite indicators of presence, that are unique and persistent 
for a user/machine. 

• Built on the familiar 

- Telephony +44 - international phone code 

- Signalling tells us this phone user is ‘online’ 

• Target Detection Identifiers 

- Started with the Internet, mobile networks too. 

- TDI is a ‘SIGINT standardised code’. 

- Not a standard managed by the ITU/ETSI. 

- Extraction from packets much more complex. 
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TDI sources 



hi5 



reddit 



BBC 



4 Hotmail 



^ Windows l i\/p 
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Target Detection Identifiers 



• 70 distinct TDI types discovered. 

• 2500 TDIs/sec (GET, de-duplicated) 

• => 200 Million per day per 1 0Gbps 

• De-dupe rate ??? 

• Cost - 250 hours per TDI 

• Automated discovery prototype 



TDI Type 


TDI Location 


User/Machine 


Yahoo-Y-Cookie 


Cookie 


User 


Yahoo-B-Cookie 


Coookie 


Machine 


Google- IK 


Request-URI 


User 


Paltalk-Nickname 


Request-URI 


User 


MS-MUID-Cookie 


Cookie 


Machine 


Google-SID-Cookie 


Cookie 


Machine 


Maktoob-MEUser-Cookie 


Cookie 


User 


Orkut-PREFID-Cookie 


Cookie 


User 


Cloob-Username 


Cookie 


User 
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□ GET data 



Report is for the 7-day period from 04/10/0S (16:20) to 11/10/0S (16:20). 
A cross-subnet threshold of < 10% has been applied. 

A mean user transmission threshold of > 10 has been applied. 



Putative selectors http://, photos/, images/, files/ are blacklisted from this plot. 



Click 0 to sort by a given column. 
0 Domain 0 Context 0 


0 Example value 


r-. Bearer 
0 counf 


. Observation 
count 


I— . Mean user transmission 
u frequency 


I— . Cross -/ 1 6 
u percentage 


facebook 


Cookie 


datr= 




8 


671 


12.98 


3.51 


facebook 


Cookie 


c user= 




8 


651 


12.09 


3.51 


facebook 


Cookie 


utrnz= 




7 


609 


12.14 


4.25 


facebook 


Cookie 


utrna= 




7 


609 


12.44 


3.56 


facebook 


Cookie 


h user= 




7 


601 


12.37 


3.74 


facebook 


Cookie 


qca= 




6 


364 


10.38 


4.97 


reuters 


Cookie 


!y= 




6 


336 


10.67 


0.24 


facebook 


Cookie 


next path= 




6 


323 


18.63 


9.18 


live 


Cookie 


MUID= 




7 


321 


10.81 


3.24 


reuters 


Cookie 


id= 




6 


312 


21.59 


0.45 


google 


URI 


£5 




7 


311 


15.02 


5.81 


reuters 


Cookie 


ss= 




6 


309 


16.83 


0.39 


yahoo 


Cookie 






7 


307 


10.76 


2.76 


yahoo 


Cookie 


d= 




6 


306 


10.60 


7.79 


youporn 


Cookie 


sid= 




S 


290 


24.90 


1.96 


youporn 


Cookie 


utrna= 




5 


282 


24.23 


1.65 


youporn 


Cookie 


utrnz= 




5 


281 


22.92 


4.60 


reuters 


Cookie 


anonId= 




6 


279 


16.22 


0.46 


youporn 


Cookie 


aca= 




5 


277 


24.40 


1.69 


yahoo 


URI 






7 


277 


31.18 


6.89 


bebo 


Cookie 


bdaysession= 




7 


275 


27.19 


2.06 


google 


Cookie 


LM= 




7 


272 


16.85 


7.31 


google 


Cookie 


ID= 




7 


271 


16.80 


3.73 


google 


Cookie 


TM= 




7 


270 


17.07 


6.57 


bebo 


Cookie 


Username^ 
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268 


27.18 


2.21 


bebo 


Cookie 


Emails 
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268 


27.67 


2.24 


yahoo 


Cookie 


1 s= 
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264 


78.61 


3.00 


google 


Cookie 
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264 


39.35 


3.82 


yahoo 


Cookie 


I£5 




3 


253 


14.24 


2.54 


yieldmanager 


Cookie 


uid= 




7 


251 


66.03 


1.01 


reuters 


Cookie 


RaptTracker= 




6 


242 


14.24 


0.48 


yahoo 


Referer 


£5 




4 


242 


17.85 


7.19 
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TDI Applications 

• Bulk store of all TDIs seen in last 6 months [MUTANT BROTH] 

• Bulk store TDI correlations (6 months) [AUTO ASSOC] 

• Bulk store TDI <-> website correlations (6 months) [KARMA POLICE] 



• Bulk store TDI vBulletin activity [INFINITE MONKEYS] 

• Bulk store TDI Social Networking Site activity [SOCIAL ANIMAL] 

• Bulk store web search requests [MEMORY HOLE] 

• Bulk store Google Earth requests [MARBLED GECKO] 
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Identifier Search 

Welcome 



IP Address Search 



Password Search 



Logged in queries logged for audit. 

Database currently contains identifiers from the period Tue Dec 25 16:26:40 2007 to Fri Jun 20 22:13:19 2008 (13.41 billion rows of as 07-3UN-08), 
Warning: data for period(s):- 
* Fri Jun 20 22:13:21 2008 - Tue Jun 24 09:33:20 2008 

is loaded, but currently unavailable for query due to index building. The rest of the database can be queried as usual during the rebuild. 

Search for Identifiers 

* if allow wildcards is ticked, % and are multi -character (bob*) and single-character (b_b@hotmai 1 .com) wildcards. 

* Q u ©r i ©s a r© a I wa y s ca se-se nsitive (bob Ghotma i l . c can T bob i 7 hat m a i l . com v b ob Ghoima r l . c om) . Th© r© i s a n opti on to a uto ma tica I [ y con vert to lo wercase . 

* For bulk queries, paste in a list of identifiers separated by newlines (one per line). 

* You can enter a mini mu m/maxi mum date for the search: default Is to search all available selectors 



MIRANDA 20135 

31 C 



Purpose | NS 



1 



Reason |demc 



P Allow wildcards 

f Con vert to lowercase before searching 

Matching Identifiers 



Ex© cut© 



The following identifiers have been found in th© MUTANT BROTH database. 

Select those that match your torgetfs) to generate a summary of to rget activity. 



TDI type 



TDI value 



□ Cha t- MS- Mess e nger 



photmail.fr 
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Identifier Search 

Welcome 



]R Address Search 



Password Search 



Logged in as 



all queries logged for audit, 

Database currently contains identifiers from the period Tue Dec 25 16:26:40 2007 to FriJun 20 22:13:19 2008 (IB. 41 billion rows of as 07-JUN-08), 
Warning: data for period(s):- 



• FriJun 20 22:1 
is loaded, but cum 

Search for Ident 


(18.41 billion rows of as 07-3 UN-08). 


rebuild. 


* if allow wildcards fe 







* Queries are always case-sensitive (bobGhotndail.com * BOB@hotmail.com t- bqb@homa il.com). There is an option to automatically convert to lowercase, 

* For bulk queries, paste in a list of identifiers separated by newlines (one per line). 

* You can enter a minimum/maximum date for the search: default is to search all available selectors 




MIRANDA 20135 

lie 



Purpose | NS 



1 



Reason |demo 



w Allow wildcards 

f Convert to lowercase before searching 



Execute 



Matching Identifiers 

The following identifiers have been found in the MUTANT BROTH database. 

Select those that match your targets) to generate a summary of target activity. 



r 



TDl type 



TD1 value 



Cha t- MS- Mess e nger 




hotmail.fr 




e 1 ft 



i -Si. Starch GGHQ 



Q 10aR2 fsif ©i PE -' : ft m bbg (J JcirV Q] im d b 1 3 sw Q cci _ ft f*? vdhjpcdia 



urrtMidliUlH Lj I -ul Authorization ksqur&d 



Lnat laentmere;, me bouree ir beo-Locaaen laencmes me omerem or me communications unit m me 



Save as CSV 
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Date Time Source IP HHFP Source IP Geo 



Identifier 



17/06/2008 17:08:44 



17/Of 



i 



WHEN 



17/06/2008 16:55:21 



17/06/2008 16:55:16 



17/06/2008 16:54:47 



17/06/2003 16:52:13 



15/06/2003 19:20:33 



Type 



Identifier Value Pass a 



6de32bb0 41.02;28.96;I5TANBUL;TR;5MMM Hi 5- Email -Cookie hotmail.com 



6de32bb0 41.0 




6de32bb0 41.9022; -87, 6726; CHICAGO; US ;5 



nihotmail.com 



6de32bbO 41.02;28.96;ISTANBUL;TR;5MMM Hi5-Email-CDokie ^^^■jihotma il.com 



6de32bbO 41.9022;-87.6726;CHICAGO;US:5MMM HiS-Email-Cookie^^^Bphotmail.conn 



6de32bbO 39.94;32.86;ANKARA;TR;5MMM H i5-Email -Cookie ^^^Wh otmail.com 



de8bdc48 33.5;36.3;DIMASHQ;SY;5MLV Hi 5- Email -Cookie hotmail.com 

I 
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Other Bulk Event Applications 

• Most events that can be associated back to TDIs: 

• File Transfer Signature (eg proof of life videos) 

• Detection by Internet profile - eg ‘Dead Letter Drop’. 

• Yahoo webcam images 

• Airline reservation confirmation emails 
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